Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks

In this article, a novel approach for training deep neural networks using Bayesian techniques is presented. The methodology allows an easy evaluation of model uncertainty and, additionally, robust to overfitting. These are commonly the two main problems classical, i.e., non-Bayesian architectures have struggle with. proposed applies variational inference in order approximate intractable posterior distribution. particular, distribution defined as product multiple multivariate normal distributions with tridiagonal covariance matrices. Every single belongs either weights or biases corresponding one network layer. layerwise posteriori variances based on expectation values, and furthermore, correlations assumed be identical. Therefore, only few additional parameters need optimized compared settings. performance new evaluated other recently developed methods. Basis evaluations popular benchmark data sets MNIST CIFAR-10. Among considered approaches, shows best predictive accuracy. Moreover, extensive provided prediction information indicate that often yields more useful estimates than comparison